SWTJC utilizes a systematic approach to instructional assessment and planning that identifies student learning outcomes, assesses the extent to which outcomes are achieved, and provides evidence of improvement based on the analysis of results.

Compliance Certification Rationale for Judgment of Compliance

IDENTIFIES OUTCOMES

SWTJC faculty identifies student learning outcomes at the course level for all courses taught and at the educational program level for both General Studies and Technical Programs. These course outcomes are consistent with state requirements provided in the Academic Course Guide Manual and Workforce Education Course Manual. Faculty also utilize advisory committees, certification and licensure standards, and program level outcomes to develop additional course outcomes. Outcomes are communicated to students in courses and to other interested parties via an individual course syllabus. All SWTJC course syllabi are available through the SWTJC website.

For the academic transfer General Studies Program, SWTJC faculty identify core educational outcomes that are consistent with the Texas Higher Education Coordinating Board (THECB) Texas Core Curriculum for undergraduate education and that reflect the faculty’s professional core beliefs and values. Faculty then select appropriate "core" courses from the THECB Lower Division Academic Course Guide Manual with content that addresses these outcomes. The General Studies Program curriculum has a set of student learning outcomes (SLO's) that , by design, directly parallel the required Core Objectives specified in the state curriculum; namely,

1. Demonstrate critical thinking skills
2. Demonstrate written, oral, and visual communication skills
3. Demonstrate empirical and quantitative skills
4. Work effectively in a team
5. Demonstrate personal responsibility
6. Demonstrate social responsibility

The General Studies Program curriculum consists of a set of core courses from different disciplines. Each core course aligns with a Foundation Component Area specified in the state plan and addresses at least three Core Outcomes; namely, nos. 1 and 2 plus at least one of the remaining nos. 3 through 6. In addition, a semester credit hour (SCH) requirement is met within each Foundation Core Area. The General Studies Program Curriculum Matrix provides an efficient way of displaying this information.

It should be noted that the student learning outcomes (core objectives) listed above are those prescribed by the Texas Legislature and the Texas Higher Educating Coordinating Board and became required beginning with the fall 2014 semester. SWTJC has begun assessing student outcomes on those core objectives. From 1999 through the 2014 summer term, SWTJC and all other Texas community colleges utilized a similar set of objectives/outcomes also provided by the state. Selected year of these “former” core objectives are presented in the following table:

In technical programs, faculty identify program outcomes with input from their respective program specific technical advisory committee. Faculty select courses from the THECB Workforce Education Course Manual with content that addresses these program outcomes. Faculty establish which courses that introduce (I), reinforce (R), and assess (A) program outcomes. A Technical Program Curriculum Map shows how these courses are linked to program outcomes. These courses, along with a minimum of fifteen credit hours of core courses selected from “Arts and Humanities,” “Science and Math,” and “Social Sciences” to compliment the technical curriculum, constitute complete curriculum plans for the technical Associate of Applied Science degrees.

After academic transfer and technical courses are selected for inclusion in programs, a common set of course outcomes is identified for each course. To encourage consistency and quality of course delivery institution wide, SWTJC utilizes a master syllabus system. For each course, faculty members create a master syllabus that contains mutually agreed-upon common components, one of which is a set course outcomes. All class syllabi provided to students contain the common components, including course outcomes exactly as contained in the associated master syllabus.

ASSESSES OUTCOMES

During the assessment process data are collected at both the course and program level. To ensure quality and consistency in assessing outcomes throughout its geographically large and diverse service area, SWTJC utilizes a variety of strategies and assessment tools. Among these are the following:

1. Use of departmental final examinations: Departmental final examinations are given in all developmental education and many core courses in English, mathematics, history, and government disciplines. These exams are developed by the faculty and designed to assess course and, where appropriate, program outcomes.

2. Use of common embedded questions within the final examinations: In courses that do not administer a departmental final exam, common embedded questions are used. These questions are developed by the faculty and designed to assess course and, where appropriate, program outcomes.

3. Use of Scantron Prosper software to collect, manipulate, and archive assessment data: In 2010, SWTJC invested in assessment technology that replaces traditional "bubble-in" answer sheets. The Prosper system has the ability to capture both objective and subjective responses and archive them on a networked server. A variety of reporting options provides faculty and administration with valuable assessment data. First, the system provides immediate feedback regarding mastery of expected learning outcomes. Prior to scoring, embedded assessment questions are linked to course and/or program outcomes. When answer sheets are scanned, linked questions are treated as subtests for which a mastery criterion is set (typically 75% or 80%). The number of students being assessed for the given outcome and the number reaching the mastery criterion are recorded in an online database, making the results immediately available to faculty and administration. Assessment data can be reported on a class by class basis or aggregated for all classes. A second advantage of the Prosper system is that assessment questions need not be limited to objective types only. Subjective questions can be included in exams, the written responses from which are scanned as an image and stored electronically. The written response can be scored immediately by the instructor and combined with objective question scoring for grading purposes. The stored image of the written response is available for later use by faculty performing a rubric-driven, juried evaluation for assessment purposes. This capability has been used to develop the Core Implementation and Assessment Plan.

4. Use of proprietary software to link assessment data directly to planning: Program and course outcome assessment data are collected and maintained in an online database that is linked to Whippleware, the college's proprietary planning software. An important feature of the software is that assessment data can be reported at the course and section levels, thereby providing assessment data based on factors such as mode of instruction (online vs. face-to-face, e.g.), demographics of students, full-time versus part-time instructors, class location, etc.

Assessment is managed differently in academic transfer and technical programs. In academic transfer programs (Associate of Arts and Associate of Science degrees), multiple methods of assessment are employed. These include:

1. Assessments are conducted in core courses while students are progressing toward the associate degree. Each core course contains certain special assignments that when completed by students, produce an artifact that can be used with an appropriate rubric for juried assessment. Of course, the assignment is graded and affects the course grade, thus encouraging the authenticity of the assessment. Twice per year, an Assessment Summit is held where random samples of core artifacts from core complete students are assessed by teams of faculty using rubrics, some faculty-developed and some which closely follow AAC&U’s LEAP Value Rubrics. Example results include those for the 2014 Written Communication assessment and the following table showing multi-year assessment of general education outcomes tied to the pre-2014 General Education Core:

2. Completion of the Educational Testing Services Proficiency Profile (ETSPP) test (formerly MAPP) which is taken by core completers and graduates - This national test is given every two years. The results from this test help determine the effectiveness of the college's general studies program and provides comparative data from other colleges. ETSPP Reports are available for 2006, 2008, 2010, 2012, and 2014.

In technical programs assessment usually takes place in a capstone course at the certificate or degree exit point. In programs that are taught in blocks, such as Automotive Technology, each block is assessed separately, and students take a national certification exam (NATEF in this case) as part of a capstone course experience. Program outcomes are assessed using a variety of methods including portfolios, written exams, and practical (hands-on) tests. Results are collected and analyzed using WhippleWare, the college's online assessment software. An example for a particular technical program (Automotive Technology) is the Program Outcomes Tracking.

PROVIDES EVIDENCE OF IMPROVEMENT

SWTJC identifies and assesses program and course outcomes to establish a basis for planned improvement of student learning. To facilitate this process, academic and technical faculty create annual unit action plans (UAP's) that document the planning, implementation, and close-out phases of the assessment process. In late spring and early summer May-June), analysis of assessment data guides faculty choice of which program or course outcomes will be targeted for improvement in the coming year. Based on this, interventions are identified, planned, and eventually implemented over the course of the college year. Toward the end of the college year, assessment-based results are analyzed and conclusions regarding improved student learning are recorded along with future actions to be taken. Such actions could include, for instance, scaling up a successful intervention to achieve wider ranging impact on learning.

Specifically, UAP's contain the following major components:

1. The program outcome and/or course outcome targeted for improvement based on assessment data.
2. A description of the intervention designed to increase mastery of the chosen outcomes;
3. The desired result (planned outcome) of the intervention
4. Links to the college's mission and goals
5. A description of the assessment method, criteria, and measures that will be used to evaluate intervention's efficacy
6. Budgetary requirements associated with the intervention
7. Findings and results related to targeted outcomes
8. Actions taken and changes made that show how the findings and results affected the program long term

During early stages of planning items 1 to 6 are completed, forming the basis of the plan. After administrative approvals are obtained, implementation can begin, generally in August at the beginning of a college year. Implementation continues through the two long terms and possibly into the summer. UAP "close-out" (items 7 and 8 above) generally is done toward the end of the spring term, but must be completed before the start of the next fall term. All phases of planning are managed using WhippleWare, the college’s online planning software. Representative examples are the Wildlife Management Program, the Administrative Information Technology Program and the Mathematics department.

At the institutional level, assessment and planning have resulted in a number of initiatives that improve student learning. Outstanding examples that are being taken to full scale include:

1. Creation of "Student Success Centers" that expand tutoring and Supplemental Instruction activities - SWTJC's TRiO program has long been noted for student successes attributable to tutoring and Supplemental Instruction. Eligibility requirements limit the number of students who can participate in this program. SWTJC applied for and received a Title V Grant to expand the availability of tutoring services and implement a college-wide Supplemental Instruction program. "Student Success Centers" were created on the college's three main campuses and staff assigned to manage tutoring and Supplemental Instruction activities. Based on positive results so far, these centers were institutionalized in Fall 2011 and are poised to make contributions to student success.

2. Creation of the "College Success Skills" course – Reports showing lack of student success in developmental education classes were the impetus for discussions that took place in 2007-2008 that lead to requiring students with two or more developmental deficiencies to take a "College Success Skills" course. The college's "Orientation" course, required of all students, failed to include many of the success strategies deemed important by the faculty. Based on the most recent cohort study, the "College Success Skills" course has shown great promise for developmental education students. See the College Success Skills COLS Annual Report.

3. By Legislative mandate, the Texas General Education Core Curriculum was revised at all state public colleges and universities; the revised 42-hr Core became a requirement in Fall 2014. SWTJC began a new assessment process for its core curriculum based upon the evaluation of the six new Core Objectives: Critical Thinking Skills, Communication Skills, Empirical and Quantitative Skills, Teamwork, Social Responsibility, and Personal Responsibility. In June of 2013, faculty from across disciplines formed the working group for the first SWTJC Assessment Summit. The main focus of that first summit was (1) to review of the preceding data and experiences with the English Department writing rubric and (2) to make the transition from the English Department rubric to the LEAP Value written communication rubric. The summit team accomplished both and spent the remainder of the two-days using the LEAP Value written communication rubric to assess over two hundred essays. At the conclusion of the summit and based on a review of the results, the working group recommended that core courses, especially first-year English place a special focus on the writing of quality thesis sentences. The second Assessment Summit was held in February 2015; results from that effort are pending.

Additional examples of planning for improvement may be viewed by clicking on "Program Planning Summaries." These are shortened versions of UAPs, giving the main components of each. Summaries are sorted and grouped by program outcome and include data from years 2006 to present. Sample of selected complete plans are also included.

Off-Site Committee Judgment and Commentary: Non-Compliance

A review of the institution’s catalog shows educational programs organized into two distinct colleges: 1) College of Liberal Arts and 2) College of Applied Sciences. In regards to the former (College of Liberal Arts), the institution provided narrative that links program outcomes with TX Coordinating Board for Higher Education outcomes. The institution identified the following six student learning outcomes for 2014 to present.

1. Demonstrate critical thinking skills
2. Demonstrate written, oral, and visual communication skills 
3. Demonstrate empirical and quantitative skills 
4. Work effectively in a team 
5. Demonstrate personal responsibility
6. Demonstrate social responsibility

A Course Guide Manual as well as a Curriculum Matrix was provided to illustrate how the College of Liberal Arts division and programs link credit courses to these six outcomes. From a review of the matrix the committee was able to examine syllabi that conveyed learning outcomes from the following departments: Communication, Math, Life and Physical Sciences, Language and Culture, Fine Arts, History, Government / Political Sciences, Sociology / Behavioral Sciences, and Optional Completion Area 2. For example, the link to the syllabus for BIO1306 master syllabus shows that this particular course addresses outcomes 1-4 from the above list. The committee was not provided a sample of departments following an organized assessment procedure.

In considering these two sources the committee could not discern how each program in the College of Liberal Arts connects assessments, results from assessments, or any changes in programs for improvement based on an analysis of results.
                       
Prior to 2014 the institution subscribed to a set of twelve outcomes required by the TX Higher Education Coordinating Board. A detailed chart was provided as evidence. It was not clear from the table how each program contributed to these findings. The committee was not able to review any sample of work to illustrate the institutional planning and evaluation processes. Assessment evidence from the 2014 Written Communication Assessment had descriptive information showing items such as what courses were assessed and the overall scores. The committee could not discern evidence of improvement based on analysis of the results from this example.

For the College of Applied Sciences (includes all technical programs), the institution employs proprietary software to record outcomes. Faculty in these programs work in consultation with advisory committees and reference the TX Higher Education Coordinating Board’s Workforce Education Course Manual to identify course learning outcomes. A technical program map was provided as an example of how the college connects courses to outcomes. The college supplied examples from the Wildlife Management program (2013-14), the Administrative Information Technology program (2013-2014), and the Mathematics Department (2012-2013). A review of the Wildlife Technology report showed three learning outcomes ranging from plant identification to writing wildlife management plans. In each case the assessment method was connected to the expected outcome and actions taken based on results were documented. For instance, outcome #2, the development of a wildlife management plan, shows that faculty will practice hands on applications of GPS and ArcMap earlier in the term and increase the number of repetitions of creating and editing shapes from 8 to 10. This intervention was designed to improve performance on a part of the course.

Taken as a whole, the committee found the institution to have an uneven distribution of work representing the standard. The committee was unable to discern clear and coherent assessments of learning outcomes for all of its educational programs. Direct assessment of student learning through evaluation and testing of student work across the spectrum of institutional programs was not a prominent feature in the response.

Institutional Response

Southwest Texas Junior College has implemented a Unit Action Plan (UAP) process, by which instructional units establish expected outcomes, implement an intervention, assess the effect of the intervention, analyze the results of the intervention, and use the results to build subsequent plans.

In response to the Off-Site Committee’s finding of “an uneven distribution of work representing the standard,” SWTJC has prepared a Unit Action Plan Summary document for instructional units. The Summary provides a much more complete picture from 2013-2015 of the planning and assessment cycle,  including expected outcomes, interventions, assessment of interventions, analysis of assessment data, and consequent planning.